A. Field of the Invention
Concepts described herein relate generally to the selective regression testing of software, and more specifically, to a static analysis process within the framework of selective regression testing.
B. Description of Related Art
Software regression testing refers to software testing that seeks to uncover regression bugs (i.e., errors) in successive versions of software. Regression bugs occur when software functionality that worked as desired in previous versions of a software product stops working or no longer works in the intended way. Typically regression bugs occur as an unintended consequence of program changes.
As more and more features are added to a software system, performing regression testing can become more expensive in terms of both human and computing resources needed to perform and monitor a regression testing cycle. For example, in some relatively complex software systems, a regression cycle may require in excess of 500 hours of computation time to test a particular version of software.
A test suite typically includes a number of test units, each of which covers some subset of the entities of the software system under test. A full regression test refers to the process of running the entire test suite regardless of what entities of the software system are changed from a previous version. It is the safest yet the most expensive approach to regression testing.
As opposed to a full regression test, a selective regression test runs a subset of the test suite, by only selecting test units whose covered entities have been changed from a previous version of the software, or fall within the changed (impacted) scope of some other changed software entities. If none of the software entities covered by a test unit is changed or impacted, it is considered safe not to run this test unit in the regression test. Since changes between two consecutive versions of a large software system are typically small compared to the whole software body, a selective regression test properly carried out can significantly reduce the required resources and lower the testing cost without impacting on the testing safety criteria.
The cost-saving advantage of a selective regression test over a full regression test comes at a price of a more complex analysis that is needed to be conducted on the software system (i.e., static analysis), the test suite (i.e., dynamic analysis) and the test unit selection algorithm (i.e., test case selection analysis). All three of these building-block processes may be equally important and can be closely integrated in order to achieve the ultimate goal of defining a precise, safe, and selective test unit set based on the software entity changes.
Static analysis of the software system under test may be done to determine an impact analysis of arbitrarily changed software entities. When a changed software entity is detected, it may be important to find out what other software entities may be affected by the given change and the software system. Based on the output of the static analysis, the test case selection analysis could ideally choose an optimal set of test units without compromising the safety of the selective regression testing.